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ABSTRACT 



Based on recommendations from a state Task Force on 



Institutional Effectiveness, the Texas Higher Education Coordinating 
Board developed a new institutional review system designed to 
identify institutional and programmatic strengths and areas of 
concern, verify institutional outcomes and improvement efforts, 
identify exemplary programs and innovative ideas, and review progress 
toward college goals. Under the evaluation process, individual 
institutions are responsible for the deployment of financial, 
personnel, and physical plant resources, while the process calls for 
yearly college self-studies to produce an Annual Data Profile and 
site visits conducted every 4 years by faculty and administrators at 
state community and technical colleges and Board staff. Specifically, 
evaluation is based on the following five critical success areas: (1) 
mission, or the institution's commitment to fulfilling the statutory 
mandates for community and technical colleges and to meeting the 
unique needs of the college's service area; (2) effective use of 
resources, assessing the commitment to policies and procedures to 
ensurt quality planning and continuous improvement of programs; (3) 
access, focusing on the commitment to serving the diverse 
educational, social, and workforce development needs of the citizens 
of Texas; (4) achievement, reviewing the commitment to attaining the 
high quality performance of students, programs, and services; and (5) 
quality, focusing on the commitment to meeting or exceeding standards 
of excellence in programs and services. A description of possible 
site visit team recommendations to colleges and appeals procedures is 
inc luded. (TGI) 
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Institutional Effectiveness 



Reviews the effectiveness of public community and technical colleges in 
achieving their statutory missions. 

Uses the evaluation results continuously to improve institutional performance and 
programs. 



Background 

In 1993. Commissioner of Higher Education Kenneth H. Ashworth appointed a Task Force on 
Institutional Effectiveness to develop a state-level evaluation and continuous improvement plan 
focusing on workforce education and academic programs at community and teclinical colleges in 
Texas. In response to the recommendations of the Task Force, the Texas Higher Education 
Coordinating Board (THECB) staff developed a new review system that: 

• Identifies institutional and programmatic strengths and areas of concern. 

• Verifies institutional outcomes and improvement efforts. 

• Identifies exemplarv- programs and innovative ideas. 

• Reviews progress toward goals established by colleges in Annual Data Profiles. 
Carl D. Perkins annual and discretionary grant applications, and OCR compliance. 

Performance Expectations 

State-level evaluation systems typically are designed to respond to the expectations of consumers 
and to goals established by the state for higher education. In Texas, expectations are placed on 
the Coordinating Board by the Legislature, Governor, and the federal government, as outlined in 
the following documents: 

• Section 6 1 .065 1 , Texas Education Code. 

• Senate Bill 642, 73rd Legislature, Regular Session, 1993. 
Federal Perkins Act (U.S. Public Law 101-392). 

Title VI of the Civil Rights Act. 

• Title IX of the Educational Amendments of 1972. 

• Section 504 of the Rehabilitation Act of 1 973 . 
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The following documents outline performance expectations for community and technical 
colleges in Texas: 

Texas Education Code, Sections 130.003 and 135.01. 

• Southern Association of Colleges and Schools (SACS) accreditation 
requirements. 

• Technical Education Program Guidelines (1993). 

Performance Measures and Core Standards for Postsecondary Technical 
Education Programs (1992). 

Texas Academic Skills Program Policy Manual (August 1993). 
Design and Purposes of State-Le vel Evaluation 

The Institutional Effectiveness process for community and technical colleges in Texas is 
designed to: 

• Accommodate the diversity of institutions. 

• Focus on outcomes. 

• Leave the process of self-assessment to the colleges. 

• Use existing databases where possible. 

• Keep the process simple. 

• Ensure integrity in the process and its results. 

The purpose of the Institutional Effectiveness process for community and technical colleges in 
Texas is threefold: 

• Contmuous improvement of community and technical colleges in response to 
state and federal goals and mandates, including workforce education and training. 
Accountability to the Texas Legislature, Governor, and U.S. Department of 
Education for public expenditures. 

Demonstration of the quality and responsiveness of community and technical 
colleges in developing a well-educated citizenry and a highly trained workforce. 

Institutional Responsibilities 

The primary focus of institutional effectiveness at the state level is the evaluation and continuous 
improvement of instructional programs. Certain ftinctions are the responsibility, first, of the 
institution's governing board and administration. Such functions include, among others: 

Deployment of college financial, personnel, and physical plant resources. 

• Management and decision-making structure and style. 

• Instructional philosophy. 
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The Institutional Effectiveness Cycle 



Data Collection 
Reporting 
Analysis 
Evaluation of Programs 
Improvement Efforts 



State-Level Evaluation Process 

The state-level evaluation process for Texas community and technical colleges consists of a 
college self-study, resulting in an Annual Dala Profile, and a four-year on-she review. 

Annual Data Profile 

The Annual Data Profile summarizes current performance data and annual progress toward 
meeting state-level goals (e.g.. Master Plan) and federal reporting requirements. The profile: 



Provides a su'mmar\* of college programs and services. 
Updates and synchronizes college and state records. 
Shows progress toward long-term goals. 
Provides a database for state-level aggregate reporting. 



The Annual Data Profile serves as a strong foundation for the on-site review by providing a 
baseline for the college to assess its effectiveness and showcase its achievements. 

Four-Year On-Site Review 

The on-site review provides a more detailed review of progress toward state and institutional 
goals. It involves a three-day campus site visit by a team composed of faculty and administrators 
from community and technical colleges and Coordinating Board staff. The team evaluates the 
critical success factors: Mission, Effective Use of Resources, Access, Achievement, and Quality. 
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Evaluation of Critical Success Factors 

According to the common elements found in state and federal goals and mandates for high 
education, the Texas state-level Institutional Effectiveness process is based on five critical 
success factors: 



The critical success factors are evaluated based on the following measures and standards: 

Mission: Review of the institution's commitment to fulfilling the statutor>' mandates for 
community and technical colleges and to meeting the unique needs of the college's serx'ice area. 

/. Success Factor: College Purpose 

A. Measure: College Purpose 

Standard: College vision and strategic plan focus on its mission 

//. Success Factor: Quality o f College 

A. Measure: SACS Accreditation 
Standard: College is fully accredited by SACS 

B. Measure: Graduate Guarantee 

Standard: College has graduate guarantees in place and published for all technical 
degrees and certificates 



Mission 
Effective Use of Resources 
Access 
Achievement 
Quality 



Critical success factors are those ^^key things that must go right 
for an enterprise to flourish and achieve its goals/' 
John Rockart, Sloan School of Management, Massachusetts 
Institute of Technology. 




C. Measure: Quality Improvement Efforts 

Standard: College fully implements at least three of the following: 

• Strategic planning 

• Program evaluation 

• Student outcome measures 

• Continuous improvement 

• Institutional effectiveness 

• Improvement plan for administering Perkins annual application 

D. Measure: Faculty Qualifications 

Standard: Faculty qualifications meet SACS standards 

///. Success Factor: Institutional Resources 

A. Measure: Educational and General Funds Costs Within Reason 

Standard: College costs are within a normal range for the size of the institution and the 
type of serv ices it provides. 

B. Measure: Financial Resources 

Standard: College financial resources are adequate to support its mission. 

C. Measure: Fiscal Accountability 

Standard: College has in place an effective accounting and budgeting system. 

D. Measure: Physical Resources 

Standard: Buildings and equipment are adequate to support college programs. 

Effective Use of Resources: Review of the institution's commitment to policies and procedures 
to ensure quality planning and continuous improvement of programs. 

/. Success Factor: Effective Use of Resources 

A. Measure: Funds Expended Appropriately 

Standard: Current funds must be expended for allowable costs. Time and effort reports 
are well-documented. 

B. Measure: Perkins-Funded Personnel 

Standard: Credentials and work experience are appropriate to assignments. 
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Access: Review of the institution's commitment to sen'ing the diverse educational, social, and 
workforce development needs of the citizens of Texas. 

/. Success Fcicior: Access !o College 

A. Measure: Access and Equity of Women and Minorities 

Standard: No significant difference in proportion of women and minorities in all 
technical education enrollment compared to overall college enrollment. 

B. Measure: Access and Equity of Special Populations 

Standard: Improvement in the representation of special populations in the institution 
relative to their representation in the service deliver)' area (SDA). Increased 
number of special populations students receiving Perkins services. 

//. Success Factor: Access to Programs 

A. Measure: Special Populations Enrollment 

Standard: Technical enrollment is comparable to total college enrollment for special 
populations. 

B. Measure: Career Planning and Counseling 

Standard: Serv ices are in place and accessible to all students. 

C. Measure: Leaniing Resource and Special Ser\'ices 

Standard: Services available to all students are accessible to special populations. 

D. Measure: Student Follow-up and Placement 

Standard: Services available to all students are accessible to special populations. 

E. Measure: Basic Skills Assessment 

Standard: Ser\'ices available to all students are accessible to special populations. 

F. Measure: Developmental Classes/Services 

Standard: Services available to all students are accessible to special populations. 

G. Measure: Office of Civil Rights (OCR) Compliance 
Standard: College fully complies with OCR requirements. 



H. Measure: Office of Civil Rights (OCR) Monitoring 

Standard: College has fully implemented previous OCR Report. 



Achievement: Review of the institution's commitment to attaining high-quality performance of 
students, programs, and services. 



/. Success Factor: Persistence. Remediation and Graduation 

A. Measure: Acceptance in Competitive Programs 

Standard: Representation by gender, ethnicity, and special populations of accepted 
students is comparable to representation in the qualified applicant pool. 

B. Measure: Remediation and Completion of Full-time Students 

Standard: 30 percent of full-time students receive a degree or certificate or transfer 
within three years. 

C. Measure: Remediation and Completion of Part-time Students 

Standard: 30 percent of all students rec ive a degree or certificate or transfer within 
five years. 

D. Measure: Remediation and Retention from Fall to Spring 

Standard: 70 percent retention of those with fewer than 1 2 Semester Credit Hours 

(SCH); 50 percent retention of those with 6-1 1 SCHi 35 percent retention of 
those with 1-5 SCH. 

//. Success Factor: Student Outcomes 

A. Measure: Course Completion 

Standard: 50 percent of students complete course. 

B. Measure: Nine Graduates Over Three- Year Period 

Standard: 1 00 percent compliance for all technical programs (except new programs). 

C. Measure: 85 percent Placement of Technical Program Graduates 

Standard: 1 00 percent compliance for all technical programs (except new programs). 

D. Measure: Technical Non-returners Employed or Pursuing Additional Education 
Standard: 50 percent of non-returners are employed or are pursuing additional 

education. 

E. Measure: Licensure Pass Rate 

Standard: 90 percent of all tested students pass licensure exams. 



/// Success Factor: Adult Vocational EducatioiVLiteracy 

A. Measure: Industr>' Contract Training 

Standard: College efforts to support the needs of local industry are evident. 
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B, Measure: .Adult Vocational Education 

Standard: Number of individuals enrolled in Adult Vocational Education (courses with 
fewer than 360 contact hours) is representati\ e of need of population in the 
sen ice area. 

C, Measure: Adult Literacy 

Standard: Number of individuals served is representative of need of population in 
ser\ ice are.a. 



Quality-: Review of the institution's commitment to meeting or exceeding standards of 
excellence in programs and ser\ ices. 

Oualitv of Academic Areas 

/. Success Factor: Quality of Academic Areas 

A- Measure: Librar>' Resources 

Standard: Quantity and quality of librar>' resources are adequate to support all 
academic and technical programs. 

B. Measure: Core Curriculum of Associate Degree Programs 

Standard: All programs have a core curriculum - 100 percent compliance. 

C. Measure: Transfer Function 

Standard: 100 percent compliance in the following: 

• At least one formalized uxitten articulation agreement in place to 
facilitate transfer of students. 

• College has studies that demonstrate the success of transfer students. 

D, Measure: College Publications 

Standard: College publications (catalog, class schedule, brochures, student handbooks, 
etc.) include all procedures, requirements, guidelines, plans, options, 
prerequisites, and policies of the college. 

E, Measure: Developmental or Remediation Programs 

Standard: Remedial courses in reading, writing, and math are present and analysis of 
remediation efforts is under way. 



//. Success ."''actor: Teaching Effectiveness 

A. Measure: Teaching Effectiveness 

Standard: College documents at least five areas in which teaching effectiveness is 
encouraged, enhanced, and/or measured, 

B. Measure: Faculty Diversity 

Standard: Gender and ethnicity of faculty are proportional to their representation in the 
student body. 

C. Measure: Professional Development Activities 
Standard: Efforts are being made to keep faculty current. 



Quality of Technical Programs 

/. Success Factor: Quality' of Programs - Part A 

A. Measure: Program Declared Majors 

Standards: Program has at least 15 declared majors during the previous three-year 
reporting period. 

B, Measure: Nine Graduates Over Three- Year Period 
Standard: 1 00 percent compliance (except for new programs). 

C. Measure: 85 percent Placement of Program Graduates 
Standard: 100 percent compliance (except for new programs). 

D, Measure: Professional Accreditation 

Standard: Program with mandatory licensure, certification, or registration requirements 
for program graduates meets standards of the respective agencies. 

//. Success Factor: Quality of Programs - Part B 

A. Measure: Compliance with THECB technical program requirements 
Standard: 100 percent compliance for AAS apd certificate programs. 

B. Measure: General Education Requirements 

Standard: 1 00 percent compliance for all associate degree programs. 

C. Measure: Faculty Support 

Standard: Number of faculty is adequate to support the program. 
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D. Measure: Equipment and Facilities 

Standard: Equipment and facilities are adequate and appropriate to support the 
program. 



E. Measure: Budget Adequacy 

Standard: Budget is adequate to support the program. 

F. Measure: Linkages and External Agreements with Schools and Universities 
Standard: Program has at least one of the following. 

• 2 + 2 (+2), 1 + 1. 

• Tech-Prep. 

• Advanced placement. 

• University transfer. 

• Inverted degree plans. 



///. Success Facior: Quality of Programs - Pari C 



A. Measure: Business and Industr>' Partnerships 

Standard: Active involvement with business'industr}' and documented evidence of at 
least two affiliations. 

B. Measure: Integrating Academic and Technical Education 
Standard: Program must include wxiting and use of computers. 

C. Measure: Employer and Student Satisfaction 

Standard: College measures employer and student satisfaction. 

D. Measure: Advisor\' Committee Membership 

Standard: Industry, gender, ethnicity, and special populations balance. 



E. Measure: Advisor}' Committee Activities 

Standard: 1 00 percent compliance. The Advisor}' Commiuee: 

• Meets at least once annually. 

• Maintains written minutes. 

• Advises on curriculum matters. 



Measure: Advisory Committee Quality 

Standard: Separate advisor}' committee for each program or closely related cluster of 
programs. 



10 



(i. Measure: Delermination of Ongoing Program Need 

Standard: Meets standard if need based on at least two of the following: 

• lndustr>' Advisor)' Committee. 

• Quality Workforce Planning Committee data. 

• Local and'or regional labor market surveys, 

• National and/or state labor market trend data. 

• Graduate placement rate exceeds 85 percent. 



H. Measure: identification of Exemplary* Program 

Standard: Program meets/exceeds all standards on this form {Exception: new 

programs qualify even if they do not meet the nine graduate and 85 percent 
placement standards) and one of the following is evident: 

• Program has received awards/commendations, 

• Program should be rated exemplar>' for reasons noted under 
^'Comments.** 
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Team Recommendations 

Following the site visit, the college receives a team report. The report contains general concerns 
and recommendations for the institution as well as the evaluation instruments for each critical 
success factor completed by the team. Technical programs reviewed receive one of the follow ing 
status recommendations: 



• Exemplary- - the program exceeds evaluation criteria. 

• Continuation - the program continues with no revisions or provisions. 

• Continuation with Improvement - the program continues, but impro\ ement is 
recommended in specific program areas. 

• Continuation with Revision the program continues with formal revisions. 

• Sunset Review - the program continues to enroll students while concerns are addressed, 
and is re-evaluated within two years. 

• Deactivation the program suspends enrollment for up to three years while concerns are 
addressed. 

• Closure - the program is discontinued and removed from the college's program 
inventory or phased out to allow currently enrolled students to complete the program. 



A ppeals Process 

The appeals process includes provisions for the reconciliation of errors of fact in either the 
Annual Data Profile or the Four- Year On-Site Review report. The college can contest 
recommendations made by the site \'isit team by providing supplemental information to 
Coordinating Board staff for further e\ aluation. 



THE GOAL OF INSTITUTIONAL EFFECTIVENESS EVALUATION IS 
CONTINUOUS PROGRAM IMPROVEMENT 
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